{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# GPT4All\n",
    "\n",
    "[GitHub:nomic-ai/gpt4all](https://github.com/nomic-ai/gpt4all) an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue.\n",
    "\n",
    "This example goes over how to use LangChain to interact with `GPT4All` models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --upgrade --quiet langchain-community gpt4all"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import GPT4All"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain_community.llms import GPT4All\n",
    "from langchain_core.prompts import PromptTemplate"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set Up Question to pass to LLM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "template = \"\"\"Question: {question}\n",
    "\n",
    "Answer: Let's think step by step.\"\"\"\n",
    "\n",
    "prompt = PromptTemplate.from_template(template)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Specify Model\n",
    "\n",
    "To run locally, download a compatible ggml-formatted model. \n",
    " \n",
    "The [gpt4all page](https://gpt4all.io/index.html) has a useful `Model Explorer` section:\n",
    "\n",
    "* Select a model of interest\n",
    "* Download using the UI and move the `.bin` to the `local_path` (noted below)\n",
    "\n",
    "For more info, visit https://github.com/nomic-ai/gpt4all.\n",
    "\n",
    "---\n",
    "\n",
    "This integration does not yet support streaming in chunks via the [`.stream()`](https://python.langchain.com/v0.2/docs/how_to/streaming/) method. The below example uses a callback handler with `streaming=True`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "local_path = (\n",
    "    \"./models/Meta-Llama-3-8B-Instruct.Q4_0.gguf\"  # replace with your local file path\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Token:  Justin\n",
      "Token:  Bieber\n",
      "Token:  was\n",
      "Token:  born\n",
      "Token:  on\n",
      "Token:  March\n",
      "Token:  \n",
      "Token: 1\n",
      "Token: ,\n",
      "Token:  \n"
     ]
    }
   ],
   "source": [
    "from langchain_core.callbacks import BaseCallbackHandler\n",
    "\n",
    "count = 0\n",
    "\n",
    "\n",
    "class MyCustomHandler(BaseCallbackHandler):\n",
    "    def on_llm_new_token(self, token: str, **kwargs) -> None:\n",
    "        global count\n",
    "        if count < 10:\n",
    "            print(f\"Token: {token}\")\n",
    "            count += 1\n",
    "\n",
    "\n",
    "# Verbose is required to pass to the callback manager\n",
    "llm = GPT4All(model=local_path, callbacks=[MyCustomHandler()], streaming=True)\n",
    "\n",
    "# If you want to use a custom model add the backend parameter\n",
    "# Check https://docs.gpt4all.io/gpt4all_python.html for supported backends\n",
    "# llm = GPT4All(model=local_path, backend=\"gptj\", callbacks=callbacks, streaming=True)\n",
    "\n",
    "chain = prompt | llm\n",
    "\n",
    "question = \"What NFL team won the Super Bowl in the year Justin Bieber was born?\"\n",
    "\n",
    "# Streamed tokens will be logged/aggregated via the passed callback\n",
    "res = chain.invoke({\"question\": question})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
